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The Center for Remediation Design and Brandeis 
University (Massachusetts) conducted a series of telephone interviews 
to develop a manual of assessment for Job Training Partnership Act 
(JTPA) youth programs. These interviews were conducted with youth 
planners in each service delivery area during August 1987. The goal 
of this project was to explore common practices , what JTPA programs 
are doing about basic skills assessment, and what the stress points 
and vital signs are. A total of 150 programs out of an original 
randomly selected sample of 205 participated. The report from the 
field is encouraging with regard to the advances made toward 
incorporating basic education skills and refining assessment in the 
absence of specific guidance or training. The assessment information 
generated from these tests was used for a combination of purposes: 
(1) to appraise basic skills; (2) to diagnose where learning should 
begin; (3) for benchmarking purposes; and (4) as a posttest to 
certify attainment. The most common practice was the use of 
standardized tests for pre and post data collection; 92% of the 
sample administered standardized tests. The most common additional 
assessment strategy was the intake interview. When asked about issues 
or problems, all staff regarded the lack of staff training in 
assessment and instruction, motivation and lack of incentives for 
participants, attendance and retention, and lack of cooperation from 
the school system as real problems. (The survey and survey results 
are included. Part 1 addresses basic skills remediation in JTPA youth 
programs and part 2 addresses JTPA assessments strategies: 
identifying issues and instruments. In addition, an appendix lists 
the 150 randomly selected programs used for this study.) (NLA) 
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Introduction 



To respond to the needs of the JTPA system, the Center for Remediation 
Design, together with Brandeis University, conducted * series of telephone 
Interviews as psrt of the development of s manual of assessment for JTPA 
Youth progress . These Interview* were conducted with youth "planners" In 
each service delivery areas administrative entity at the end of August 
19B7. Simply, the goal was to explore common practices, what most JTPA 
programs were doing about basic skills assessment now, and what were the 
stress points and vital signs. A total of 150 programs out of an 
originally randomly selected sample of 205 participated. 

Overall, the report from the field is encouraging, at times even 
surprising, with regard to the advances made toward incorporating basic 
education skills and refining assessment in the absence of specific 
guidance or training. For example, nearly 702 of the programs sampled 
provided basic skills remedistion both in summer and during the school 
year, 282 during the summer only. Most programs used s variety of 
instructional techniques, but among the most impressive findings is that 
more than 70% of the programs now use computers as teaching tools, nearly 
752 employ genuine individualized competency-based techniques and nearly 
602 tied basic skills instruction to work experience; thereby modeling some 
of the most critical elements of effective programs for at-risk youth. 
Eighty-five percent of the programs explained that basic skills remediation 
was a function of their JTPA youth employment competency system and when 
asked how "orapetency gains were measured nearly 252 reported using grade 
level advances followed closely by 21 X reporting criterion — referenced or 
functional skill gains (often to supplement not replace grade level 
scores). Ot'.iers reported defining attainment through some combination of 
grade level scores and G.E.D. test scores. 

The single most revealing question with regard to who's doing what 
with assessment — both in the summer and year round programs — turned out 
to be "what tests(s) do you use?" Of those programs administering 
standardized tests (922 of the sample), the following emerged es the most 
commonly used: 



Test of Adult Basic Education (TABE) 
California Achleveaent Test (CAT) 
Wide Range Achievement Test (WRAT) 
Adult Basic Learning Exaaination (ABLE) 



Dsed by aore than 39% of programs 
Deed by acre than 222 of programs 
Dsed by nearly 172 of programs 
Used by nearly 102 of programs 



The assessaent information generated froa these tests was used for a 
coablnation of purposes including: 1) to appraise basic skills in order to 
sort youth and tsslgn thea to appropriate programs (352 of prograas); 2) to 
diagnose where learning should begin within a defined level (702 A 
programs); 3) for benchmarking progress (312 of programs); and 4) as a 
post-test to certify attainment or gain (662 of programs). One can *rfer 
that the most common practice is the use of standardized tests for pre- and 
post-data collection. The most common additional assessment strategy used 
was the intake interview which was used by 452 of the respondents. When 
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asked about issues or problems in implementing effective basic skills 
remediation programs under JTPA, without exception all practitioners 
digressed from the interview protocol to indicated that they regarded the 
lack of staff training in assessment and instruction as a real problem, the 
next most often mentioned problems included "motivation and lack of 
incentives for participants/ "attendance end retention" and "lack of 
cooperation from the school ^etem.* These four problems are also 
regularly raised by participants attending The Center for Remediation 
Design's Institutes on Basic Skills. 



The survey was conducted during August 1987. The summary that follows 
reports on 150 programs out of an originally randomly selected sample of 
205. (This sample was developed by taking every third SDA on an 
alphabetized list of approximately 610 SDA administrative entities). The 
attached appendix shows the distribution of the sample together with the 
number of responses state by state. If no bias was Introduced by the 
sample not being completed, the sample size is probably adequate for the 
purpose intended (with an error of not more that 8% at the 95Z level of 
confidence). There does not seem to be any obvious variation of responses 
among states. Many states had only one respondent and comparison is 
therefore undependable. The one broad comment which can be made is that 
the variation of responses within states depends mainly upon the number of 
respondents within the state. 

The interviewers asked to speak to the person in charge of the SDA's 
youth program. First contacts were not usually well informed about the 
programs in force. Further referrals (often as many as seven ) proved to be 
of greater help and were more enthusiastic about programing efforts* As a 
rule, JTPA program operators tended to have more information than the SDA 
or the PIC contacts* 



Methodology 




The Survey and the Results 
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BASIC SKILLS REMEDIATION IN JTPA YOUTH PROGRAMS 



1« Do you provide basic skills remediation for JTPA youth? 

yes, fiuoser only (I IB) 

yes, school year only 

yes, both summer and school -year 

no 



69.3% of the programs sampled provide basic skills remediation both in 
summer and during the school year, 2BX during the sumer only and IX during 
the school year only, (One response was not available,) 

2. Who is served in your program(s)? (Check all applicable .) 

in-school youth 

dropouts 

______ high school graduates 

Programs typically serve youth who are still in school, together with 
others no longer in school - this combination represents 92? of the sample. 
Other target groups for service were all encompassing : 

- 682 of the sample had programs which served 'in school* youth, 
dropouts, and high school graduates; 

- 16. 6Z served *in school* youth and dropouts; 

- 21 served 'in school* youth and high school gradates; and 

- 7,3* served * in school* youth only. 

One response was not available and the remaining 2.6X of the sample offered 
programs to dropouts and high school graduates only, 

3. Describe your program's instructional technique. (Check all 
applicable.) 

group instruction 

^ individual /self-paced 

competency-based 

_ computers are used as teaching tools 

instruction is specifically tied to work experience 

instruction is specifically tied to skill* treining 
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Most programs used a variety of Instructional techniques varied by 
program and client need such as: 

- 70. 72 used computers as teaching tools (to any extent); 

- 742 used competency based techniques; 

- 57.32 tied instruction to work experience; 

- 53.32 tied instruction to skills training; 

- 73.3* used individual/self paced techniques; and 

- 73.32 used group instruction. 

The most common combination of techniques was to use all of them - 
this was the case for 24.72 of respondents. The next most common 
combination of techniques was to use group instruction, individual/self- 
paced instruction, competency based instruction and computers as teaching 
tools - this combination was used by }.32 of respondents. 

4. How would you rate the results of your program? 

Excellent Good Fair Poor 



Perhaps predictably, respondents rated their program results very 
highly. (Subjective judgments of "excellent", "good" etc. were used:) 

- 27. 62 claimed to have excellent results; 

- 57,7% reported good results; 
5.3? reported fair results; 

- one respondent (0.72 of the sample) reported poor results; and 

- 8.72 of the sample gave no response. 



5. How is your remediation program funded? 

JTPA 82 

JTPA IIA 

JTPA I IB 

other (please be specific) 

Funding for the programs most typically came from JTPA exclusively and 
were usually derived from a combination of sources: k 

- 2.72 were funded from JTPA 82 only ; 

- 8.72 were funded from JTPA IIA only ; 

- 2S.72 were funded from JTPA II S onlp ; 

- 29.32 wsre funded from a combination which included JTPA 82 ; ■ 

- 58.72 were fund from a combination which included~JTPA IIA; 

- 862 were funded from a combination which included JTPA IIB; and 

~ the most common combination was that of JTPA I'J^ and JTPA IIB which 
was used by 30.72 of respondents. 

6. Is your remediation program linked to a JTPA youth competency system? 

If yes: are competency gains measured by grade level scores? ~ 
Functional skill gains? CED test? 



f; 



- 5 - 



85% of the programs were linked to a JTPA youth competency system . Of 
these, the most common procedure for defining outcome or attainment was: 

- grade level scores - 24.32; 

- functional skill gains - 21.32; and 

- a combination of grade level scores and GED test - 13.32. 
No other option was used by more than 10Z of the sample* 

7. What do you see as the three biggest problems in providing remediation 
to youth in your programs? (Topics to be covered in the paper.) 

Most respondents mentioned more than one problem in providing 
remediation. The most often mentioned problems were "motivati on and type 
of incentive programs" and "remediation problems and attendance* (32.72 of 
respondents - 142 mentioned only this problem). Other significant 
problems were: 

- 16.7? mentioned ^role clarification of JTPA vs school 
responslbllitles~f or youth" ; 

- 15.3Z mentioned " lack of cooperation from school system "; 

- 13.32 mentioned recruitment ; 

- 122 mentioned rural county problems ; and 

- 10.72 mentioned transportation . 

No other problem was mentioned by more than 102 of respondents. 
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Part II 

JTPA ASSESSMENT STRATEGIES: IDENTIFYING ISSUES 4 INSTRUMENTS 



8. Do you provide formal testing for youth in remediation in IIA? in 
I IB? (Standardized.) 

921 of the programs provided formal testing for youth in remediation. 

9. If you administer a formal test(s) what do you use in IIA? 
in IIB? (List all that apply.) 

Of those JTPA programs which administer formal testa themselves (as 
opposed to the school system) the following emerged as the most commonly 
used : 

- TABE is used by 39.3% of programs; 

- CAT is used by 22. 7X of programs; 

- WRAT is used by 16.7? of programs; 

- ABLE is used by 9.3X of programs; and 

- 7. '3% of tests used were self made . 

None of the other tests mentioned were used by more than four 
respondents (2.7X of the sample). 

10. How do you use assessment information? 

Test to sort to diagnose for progress checks credentlallng 



Assessment information for a combination of purposes by most programs 

were: 

- 34. 7S used it to tort youth into groups (appraisal); 

- 68.7% used It to diagnose where learning should begin within a 
defined level; 

- 30.71 used It for progress checks (benchmarking); and 

- 661 used It for certifying attainment . 



11. What other assessment strategies besides tests do you use? 

Intake interview? Performance reviews (behavior observation)? 
Product development? Other? 



The most common additional assessment strategy used was the intake 



interview* which was used by 44. IX of respondents » None of the other 
strategies, or combination of strategies was used by more than 10 J of 
respondents* 



12. Do you use information from other sources? If yes» what tests? What 
sources (i.e., schools)? 



Information from schools was the only other commonly mentioned source 
of information. No other source was mentioned by more than one program - 
95, 32 of respondents mentioned school as an information source. 
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Appendix* 
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* A randomly selected sample of 150 programs was used for this study. 
This was developed by taking every third SDA on an alphabetized list of 
approximately 610 SDA administrative entities* 
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